-LRB- CNN -RRB- -- Ukrainian President Viktor Yanukovich pledged Thursday to make his country `` a bridge between East and West '' after being sworn into office in Kiev .

Yanukovich , who was ousted from office in 2004 in the pro-West `` Orange Revolution , '' beat Prime Minister Yulia Tymoshenko in a bitterly fought election earlier this month that has left the country deeply divided .

Yanukovich enjoys support among the Russian-speaking people living in eastern Ukraine , while Tymoshenko is mainly supported by the Ukrainian speakers in the western part of the country . There is concern Yanukovich will move the former Soviet state more closely into Moscow 's orbit and many fear further political turmoil .

But in his inaugural address , Yanukovich said it was in Ukraine 's interest to build close relations with Russia , the European Union and the U.S.

`` Being a bridge between East and West , an integral part of Europe and the former Soviet Union at the same time , Ukraine will choose a foreign policy that will allow our country to get the most out of the development of equal and mutually beneficial relations with Russia , the European Union , the United States and other countries that influence development in the world , '' he said .

Yanukovich also said he would put Ukraine on a `` path of accelerated development '' to save Ukraine from socio-economic collapse and called for cooperation between president , parliament and government .

In one of his first action as presidents , Yanukovich said he had reduced the costs of maintaining a presidential administration by 20 percent . The money saved would go to the fifth of Ukrainians living in poverty , according to a statement from his office .

Yanukovich was declared the winner of the presidential election in 2004 , but the legitimacy of the vote was questioned and he faced accusations of stealing the race .

Yanukovich 's win was annulled following the Orange Revolution uprising , with former President Victor Yushchenko winning the revote .

Tymoshenko stood alongside Yushchenko during the Orange Revolution , but later political infighting broke down their alliance . In a recent interview with CNN , Yanukovich attributed his dramatic comeback to the current economic situation , saying people voted for him because they believe he can improve the economy .

`` I want to tell the people of Ukraine ... we will repay the arrears of wages and pensions , what the current government never managed to do , '' he said Thursday . `` Given the dire situation in public finances , we will do this by reducing the costs of -LRB- the -RRB- bureaucratic system . We will start with ourselves . ''

@highlight

Yanukovich pledges to make country `` a bridge between East and West ''

@highlight

Yanukovich won bitterly contested presidential election earlier this month

@highlight

President returns to power after being ousted in 2004 Orange Revolution

@highlight

Yanukovich pledges to put Ukraine on `` path of accelerated development ''